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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 
13, 2008 has been entered. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

3. Claims 1,4-17, 28-32, 36, 39-52, and 63-67 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over the combination of Acharya (US 2002/0101524 A1, 
hereinafter referred to as Acharya002) and Yeo et al. (US 6,738,509 B2). 
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Regarding claims 1 and 36, Acharya002 discloses a color image processing 
system comprising: 

act A: converting a single first image data from a first color space into a second image 
data that corresponds to a second color space (as depicted in figure 1 A, image data 
converted from 8-bit Bayer color space to 24-bit RGB. Figure 1 B depicted the 
conversion from 24-bit RGB space to 24-bit YCrCb. "Once each pixel of the image is at 
a full RGB color resolution (either through color interpolation or by virtue of having been 
captured at full color resolution), another process known as color space conversion may 
be performed. Depending on the rendering device or particular application for which the 
full color image is destined, the RGB "color space" (of an interpolated image, see FIG. 
1(a)) may need to be converted ..." in paragraph 10, line 1 . The conversion equations 
from RGB color space to YCrCb color space are listed in paragraph 28); 
the single first image data includes a first pixel and a second pixel, wherein the first pixel 
corresponds only to a first color component and the second pixel corresponds only to a 
second color component that is different from the first color component (as depicted in 
figure 1A for the 8-bit Bayer pattern "The Bayer pattern (see FIG. 1(a)), when three 
color planes are sub-sampled sampled, is characterized by including on a first row of an 
image, alternating pixels of Red and Green, and then on a next row alternating pixels of 
Blue and Green. Thus, the Bayer pattern for the first four rows of pixels (with each pixel 
typically an 8-bit value) is as follows (with the rows thereafter repeating the same 
pattern): 

G RG RG R . . . 
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BGBGBG. . . 
G RG RG R . . . 
BGBGBG.... 

As a result, each pixel location contains a value for a single color only" in 
paragraph 7, line 14). 

Acharya002 discloses that color space can be converted to various desired color 
spaces. Acharya002 does not explicitly disclose image processing on the selected 
color space. 

Yeo, in the same field of endeavor of color image processing ("compressing 
multi-spectral images in different spectral spaces and transforming images from one 
spectral space to another" at column 1, line 14), teaches image processing under 
various color spaces "a multi-spectral image is a collection of two or more monochrome 
images of the same scene. Multi-spectral images can be described in any one of a 
plurality of known spectral or color spaces. For example ... RGB color spectral space 
... CIE (Commission Internationale de L'Eclairage) L*a*b*, CIE XYZ, CIE L*u*v*, CIE 
YUV, CMY . . . CMYK . . .YCbCr, YIQ, HIS, and HSV ..." at column 1 , line 20. As 
depicted in figures 2 and 3, numeral 105 is the codec processing in the second color 
space to form a processed image data (claim act B) and numeral 1 16 to convert the 
processed image data to the third color space in figure 2 (claim act C) or to the first 
color space in figure 3, numeral 304 (claim act C), or to color space 2 in figure 3, from 
numeral 102 to numeral 116 (claim act C), where the conversion method used, figure 4 
or figure 5 of Yeo, is different from the method Acharya200 used for act A. 
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It would have been obvious at the time the invention was made to one of ordinary 
skill in the art would have been motivated to include the color image processing system 
Acharya002 made, with color space data processing as taught by Yeo, because there 
are practical applications for color space and data processing: "Most JPEG images are 
compressed in the YCbCr color space. For MPEG-1 this is the only allowed color 
space" at Yeo column 2, line 36, and "image editors or rendering devices such as 
printers do not use RGB color space in manipulating or rendering images ..." in 
Acharya002 paragraph 10, line 8. 

Regarding claims 4 and 39, the first color space is an RGB raw space (as 
depicted in Acharya002 figure 1 A 8-bit Bayer pattern "In digital still and video cameras 
and certain other imaging devices, raw images are first represented as rectangular row 
and column of pixels with each pixel having the intensity value of a particular color only. 
In the case of RGB (Red, Green and Blue) sub-sampling imaging devices, images are 
obtained and stored in a particular pattern. FIG. 1(a) shows are such pattern ..." in 
Acharya002 paragraph 7, line 6). 

Regarding claims 5 and 40, the second color space is a single color component 
color space (discussed in claims 1 and 36, the color space can be described in any one 
of a plurality of known color spaces. The second color space is a single color 
component color space). 
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Regarding claims 6-7, and 41-42, the second color space is a multiple color 
component color space (discussed in claims 1 and 36, the color space can be described 
in any one of a plurality of known color spaces. See also "... color space 2 may be 
YCbCr color space. Other color space combination are also possible ..." at Yeo column 
4, line 49). 

Regarding claims 8-10 and 43-45, the third color space is a single color 
component color space (discussed in claims 1 and 36, the color space can be described 
in any one of a plurality of known color spaces. The third color space can be a single 
color component color space, a multiple color component color space, or RGB raw 
space, or YCbCr space). 

Regarding claims 1 1 and 46, act A further comprises using one or more 
temporary buffers to store the second image data (as depicted in Acharya002 figure 6, 
numerals 71 1 and 734 for storages. See also "... a system unit having a central 
processing unit (CPU) and associated volatile and non-volatile memory, including all 
RAM ...CD-ROM drive ..." at Yeo column 10, line 42). 

Regarding claims 12 and 47, act B further comprises using one or more 
temporary buffers to store the processed image data (discussed in claims 1 1 and 46). 
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Regarding claims 13 and 48, act B further comprises image data compression 
(as depicted in Yeo figure 9 for data compression process). 

Regarding claims 14, 28, 49, and 63, act A further comprises performing a color 
interpolation for converting each pixel that is associated with the first image data from a 
single color component to multiple color component to form a corresponding 
interpolated pixel (as depicted in Acharya002 figure 1 A from 8-bit Bayer pattern to 24-bit 
RGB. For the performance of color interpolation "...To interpolate the Green color 
component G'01 , one method of color interpolation would average together adjacent 
Green color plane associated pixel values which are, for instance, GOO, G1 1 and G02. 
Thus G'01 might be determined by computing (G00+G1 1 +G02)/3 ..." in Acharya002 
paragraph 27, line 15. This interpolation is the nearest neighbor interpolation (claims 28 
and 63). More complete discussion can be found in paragraph 27). 

Regarding claim 15 and 50, applying a conversion equation to each interpolated 
pixel, wherein the conversion equation is selected based on the second color space 
("For instance, when converting from RGB color space to the YCrCb color space, a set 
of conversion equations are defined such that for a given pixel location (ij) each 
component Y, Cr and Cb is a linear combination of R, G and B components ..." in 
Acharya002 paragraph 28, line 1 . More complete discussion can be found in paragraph 
28). 
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Regarding claims 16 and 51, act A further comprises applying a conversion 
equation to each pixel, wherein the conversion equation is selected based on the 
second color space (as depicted in Acharya002 figure 2, shows the steps of color space 
pixel by pixel conversion. Detail discussions about the conversion formula used can be 
found from paragraphs 27 to 29). 

Regarding claims 17 and 52, color interpolation further comprises deriving 
missing color components for each pixel from the pixel's neighboring pixels, wherein the 
neighboring pixels contain the missing color components (as discussed in claims 14 and 
49, G'01 = (G00+G1 1 +G02)/3 where G'01 is the missing color components, GOO, G1 1 , 
G02 are neighboring pixels of G'01 . Reference Acharya002 figure 1 A for details). 

Regarding claims 29 and 64, act C further comprises re-mapping each pixel of 
the processed image data into the selected color space (as depicted in Acharya002 
figure 1C, the 24-bit YCrCb data decimated to 12-bit YCrCb). 

Regarding claims 30 and 65, act C further comprises applying a conversion 
equation to each pixel of the processed image data, wherein the conversion equation is 
selected based on a selected color space from the set of color spaces (as depicted in 
Yeo figure 4, numeral 402 or figure 5, numeral 502 shows the color space conversion 
equations). 
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Regarding claims 31 and 66, after applying the conversion equation, re-mapping 
each pixel of the processed image data into the selected color space (as discussed in 
claims 30 and 65 conversion equation applies to each pixel; claims 29 and 64 to re-map 
each pixel of the processed image data into the selected color space). 

Regarding claims 32 and 67, re-mapping includes dropping undesired color 
components (as depicted in Acharya002 figure 1C and figure 4, the undesired color 
components of Cb and Cr are dropped so the file size can be reduced). 

4. Claims 18-27, 33-35, 53-62, and 68-70 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over the combination of Acharya002 and Yeo as applied to claims 1 
and 36 discussed above, and further in view of Rashkovskiy et al. (U.S. 6,252,577 B1 ). 

Regarding claims 18 and 53, the Acharya002 and Yeo combination discloses 
color space interpolation. The Acharya002 and Yeo combination does not explicitly 
disclose other possible color space interpolation acts and the filters. 

Rashkovskiy, in the same field of endeavor of "digital image processing" at 
column 1 , line 5, discloses the details of interpolation from a single color component to a 
multiple color component: "A method according to an embodiment of the invention 
operates to partially determine the scaled image, leaving missing components in the 
second and third sets of pixels. The partial determination may be done using linear 
mathematical operators that combine interpolation and low pass filtering and are applied 
to selected regions of the original image" in column 2, line 30. Furthermore, "the well 
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known Hamming filter works particularly well and is relatively simple to implement" at 
column 6, line 66. "each pixel in the original image is converted from having a single 
color component into one pixel having multiple color components ... by interpolating the 
additional color components of a pixel based on the intensities and colors of its 
neighboring pixels ..." at column 1, line 37. Thus, the interpolation is based on nearest 
neighbor interpolation. The said selected region of the original image is a 3X7 array 
with 21 Bayer pattern coefficients (at column 4). The interpolation "may take on a larger 
or smaller size. In addition, the array need not have an odd number of coefficients on 
each side" at column 5, line 1 . Equation (1 ) (at column 5) is the interpolation formula 
used where cg(ij) is the weighting coefficient. For a 1X3 region, the missing color 
component G(2) can be derived from the pixel's closest previous G(1) and next pixel 
G(3) in a horizontal direction (act P for claims 18 and 53). G(2) can be the averaged 
output if the weighting factor eg is 0.5. Otherwise, G(2) is the weighted output. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to implement the color space transformation methodology of the 
Acharya002 and Yeo combination, with more single/multiple color interpolation options 
as taught by Rashkovskiy, in order not only to explicitly describe the steps of color 
interpolation processing for "efficiently downscaling images and for transferring images" 
at Rashkovskiy column 1 , line 6, but also to avoid "any aliasing effects" at Rashkovskiy 
column 7, line 31. 
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Regarding claims 19-20 and 54-55 (discussed in claims 18 and 53, for the 
averaged output and the weighted output). 

Regarding claims 21-22 and 56-57, act S further comprises averaging pixels 
corresponding to each missing color component from the previous line of pixels (With 
proper selection of the size of target area, Rashkovskiy Equation-1, at column 5, and 2, 
Equation-2, at column 6, can be used to derive averaged or weighted pixel values 
corresponding to each missing color component from the previous line of pixels). 

Regarding claims 23 and 58, the fixed number is based on missing color 
components from previous frames ("... missing components, however, may now be 
readily computed using conventional interpolation techniques as will be apparent to 
those of ordinary skill in the art ..." at Rashkovskiy column 4, line 1 . Thus, the fixed 
number is based on missing color components from previous frames is one acceptable 
alternative method). 

Regarding claims 24 and 59, low-pass filters used ("next step is to determine the 
filtering that will be applied to the selected region to generate the G component of the 
target pixel in the scaled image ... Equation-3 ..." at Rashkovskiy column 6, line 13. 
"Any one of a number of different filters can be used for h(j), but the well known 
Hamming filter works particularly well ..." at Rashkovskiy column 6, line 64. Where 
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Hamming filter is a low pass filter and variable G'(j) in equation-3 is the value after 
interpolation). 

Regarding claims 25-27 and 60-62, using filters before, after, before and after 
performing the color interpolation (as discussed in claims 24 and 59, performing filter 
after interpolation. However, Equations-4 and 5 of Rashkovskiy at column 6, perform 
the filter before interpolation for G(i) is the value before interpolation. Without departing 
from the essence of Rashkovskiy's methodologies, filters can be applied before and 
after performing the color interpolation). 

Regarding claims 33-35 and 68-70, using filters before, after, before and 
after dropping undesired color components (discussed in claims 25-27 and 60-62 this 
filter can be applied before, after, and before and after color interpolation. Without 
departing from the scope and spirit of Rashkovskiy's methodologies, this filter can also 
be applied before, after, and before and after dropping undesired color components). 

Response to Arguments 

5. Summary of Applicant's Remarks: 

"Thus, independent claim 1 includes converting a single first image data that 
includes a first pixel corresponding only to a first color component and a second pixel 
corresponding only to a second color component that is different from the first color 
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component. Applicant respectfully submits that Yeo fails to disclose this expressly" at 
response page 22, line 1. 

"Thus, amended claim 36 includes converting the processed image data to a 
third image data of the second color space, but using a conversion method that is 
different than the conversion method that is used in the initial conversion of the first 
image data. Applicants respectfully submit that Yeo fails to disclose at least this 
expressly recited element" at response page 23, line 1 1 . 
Examiner's Response : 

Applicant's argument is moot in view of the new grounds of rejection advanced 
herein above. Specifically, the combination of Acharya (US 2002/0101524 A1, 
hereinafter referred to as Acharya002) and Yeo et al. (US 6,738,509 B2) reference now 
teaches the claimed Bayer color space. Refer to the rejections above. 

Conclusion 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eueng-nan Yeh whose telephone number is 571-270- 
1586. The examiner can normally be reached on Monday-Friday 8AM-4 :30PM EDT. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on 571-272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Eueng-nan Yeh 
Assistant Patent Examiner 
2624 

/E.Y./ 



A/ikkram Bali/ 

Supervisory Patent Examiner, Art Unit 2624 



